1. Field of Invention
This invention relates to the processing of a digital representation of a signal from an optical image sensor, for example, the signal from the optical image sensor used in a digital image capture device such as a digital camera. More particularly, this invention relates to a system and method of digital image processing in which Black Level Correction (BLC) processing is until after Noise Reduction (NR) processing.
2. Discussion of Related Art
Signals produced by an optical image sensor used in an image capture apparatus, such as a digital camera, based on Charge Couple Device (CCD) or Complementary Metal Oxide Semiconductor (CMOS) technology, or, for that matter, any other optical image sensor technology, usually have a measured value when no light is impinging on the surface of the sensor. This signal output level is called “black level”. Black level is usually, but not necessarily, approximately the same for all pixels in the image. In an effort to cause the image pixels to be at a 0 value when the optical image sensor is in total darkness, one of the earliest processes executed on the digital representation of the optical image sensor's analog output signal by the image data signal processor of the image capture apparatus, is the estimation and removal of the black level present in the image data signal. Removal is effected by subtracting the estimated black level from all the pixel values in the image. This operation is called Black Level Correction (BLC).
FIG. 1 is a graph showing the response of a typical optical image sensor to increasing light impinging on its surface, before the execution of a BLC processing operation. The response is shown after conversion from an analog to a digital signal.
FIG. 2 is a graph showing the response of a typical optical image sensor to increasing light impinging on its surface, after the execution of a BLC processing operation. The response is shown after conversion from an analog-signal to a digital signal. The black level of FIG. 1 has been corrected to black level of value 0 in FIG. 2.
In addition to black level, Images produced by signals from an optical image sensor almost always contain noise. The amount of noise in the image depends on the conditions under which the image was captured. These conditions include the amount of gain provided by the amplifier receiving the output of the optical image sensor signal and connected to the image capture apparatus' analog to digital converter. An image captured under high gain conditions is referred to as a “high ISO image”, while an image captured under low gain conditions is referred to as a low ISO image. Exposure setting, for example lens aperture size and shutter speed, also affect the amount of noise in the captured image. The higher the light level on the surface of the optical image sensor and the longer the exposure time during the image capture process, the higher the level of the sensor's output signal and the lower the noise level in the captured image signal. The lower the light level on the surface of the optical image sensor and the shorter the exposure time during the image capture process, the lower the level of the sensor's output signal and the higher the noise level in the captured image signal. A major component of the noise level, called “shot noise” usually behaves as a square root function of the optical image sensor's output signal level. However, even in the absence of light, which produces a dark image sensor output signal, there will still be a certain amount of noise present. This noise is referred to as “black noise”, and is electrical in nature.
FIG. 3 is a graph showing typical noise level on the output signal from an optical image sensor, as a function of image data signal level, before the execution of a BLC processing operation.
FIG. 4 is a graph showing the typical noise level on the output signal from an optical image sensor, as a function of image data signal level, after the execution of a BLC processing operation. The black level of FIG. 3 has been corrected to a black level of value 0 in FIG. 4.
A BLC processing operation, even when performed accurately, can result in negative pixel values. This is caused mainly by the black noise which may be present in the image. In most image data signal processors, the negative pixel values are clipped to 0. This causes an unbalanced noise distribution in dark areas of the image. Thus, once the noise is removed, either by an edge preserving low pass filter, median filter or almost any other Noise Reduction (NR) filter, the signal level in the dark areas of the image will be higher than the true signal level. Therefore, because of noise clipping that can occur during a BLC processing operation, black areas in the image may not be as black as they should be, thus producing an image with lower contrast. Further, noise clipping during a BLC processing operation increases the non-linearity of the sensor in dark areas. After applying white balance correction, this non-linearity may be translated into a false color cast in the dark areas of the image. The problem of noise clipping during a BLC processing operation becomes even more severe when an image is a high ISO image, because the image contains high noise levels.
FIG. 5 is a graph showing the typical black noise level distribution on the output signal from an optical image sensor, following the signal's conversion from an analog-signal to a digital signal, before the execution of a BLC processing operation.
FIG. 6 is a graph showing the typical black noise level distribution on the output signal from an optical image sensor, following the signal's conversion from an analog-signal to a digital signal, after the execution of a BLC processing operation that does not cause black noise clipping.
FIG. 7 is a graph showing the typical black noise level distribution on the output signal from an optical image sensor, following the signal's conversion from an analog-signal to a digital signal, after the execution of a BLC processing operation, that causes the signal's black noise level distribution to be clipped to signal values above pixel value 0.
FIG. 8 is a graph of the response of a typical optical image sensor to light impinging on its surface, after the execution of a BLC processing operation that does not cause the optical image sensor signal's black noise level distribution to be clipped to signal values above pixel value 0, such as shown in FIG. 6, as compared with the response obtained if a BLC processing operation that causes such clipping, such as shown in FIG. 7, is used. Note the increase in nonlinearity in the dark areas of the image when clipping occurs.
BLC clipping can be avoided by adding bits to the image processing path of an image capture apparatus, thus allowing the accurate processing of both positive and negative digital values without clipping. However, the cost or image quality impact of doing so can be prohibitive, especially for many digital camera applications. This is because this approach requires assigning a sign bit to each image pixel being processed. Thus, either the amount of memory used by the digital camera needs to be increased, to account for the extra sign bit, or the amplitude dynamic range of the digital camera's image pixels needs to decrease, as an existing bit is reassigned to serve as the sign bit. In the former case, manufacturing cost becomes a limiting factor. In the latter case, the loss of dynamic range can make the digital camera noncompetitive in terms of image quality.